Object search and navigation method and system

ABSTRACT

A computer implemented method of presenting visual content items, comprising: grouping the visual content items according to predefined similarity rules relating to visual characteristics of the visual content items such that each group has a range for the number of its members; selecting a representative visual content item for each group; presenting the representative visual content item of each group that has a minimal number of members above a predefined threshold; and optionally presenting the visual content items alongside the representative visual content items. The grouping may be carried out by using predefined color groups; by using predefined shape groups; or using Keypoints of the visual content items, and relate to groups of human faces, product images and landscape images.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/078,789 filed on Jul. 8, 2008, which is incorporatedherein by reference.

BACKGROUND

1. Technical Field

The present invention relates to searching, presenting and navigatingwithin a list of objects, and more particularly, to navigating usingcontent items.

2. Discussion of Related Art

Electronic shopping is becoming evermore elaborate and versatile, yetusers are confronted with ever growing range of products that they maywish to choose from. Current problems with electronic commerce are:Overload—A shopper will need to go through hundreds or even thousands ofpages in order to get some orientation on the product selection;Requires Familiarity—The are many criteria for narrowing down theselection, yet some of them require the shopper for prior familiaritywith the category which could be different has a photo been presented;No sub division—There is no way to really those thousands of deals tomajor subgroups; No Pareto—There is no for the shopper to focus hisefforts on the major products rather than in ancillary products that mayfind themselves to the higher parts of the pages; Redundancy—in manycases tens of deals with the same offering are presented. Thesedisadvantages may shopping experience tedious.

BRIEF SUMMARY

Embodiments of the present invention provide a computer implementedmethod of presenting a plurality of visual content items, comprising:grouping the visual content items according to predefined similarityrules relating to visual characteristics of the visual content itemssuch that each group has a range for the number of its members;selecting a representative visual content item for each group; andpresenting the representative visual content item of each group that hasa minimal number of members above a predefined threshold.

Accordingly, according to an aspect of the present invention, there isprovided a computer implemented method, further comprising presentingthe plurality of visual content items alongside the representativevisual content items.

Accordingly, according to an aspect of the present invention, thegrouping may be carried out by using predefined color groups; by usingpredefined shape groups; or using at least one keypoint of the visualcontent items, and relate to groups of human faces, product images,landscape images.

Embodiments of the present invention provide a data processing systemfor analyzing and presenting a plurality of visual content items,comprising: a mediator server comprising a graphical user interface, themediator server connected via a communication link with a user and witha plurality of sources holding the visual content items, and arranged togroup the visual content items according to predefined similarity rulesrelating to visual characteristics of the visual content items such thateach group has a range for the number of its members; and to select arepresentative visual content item for each group, wherein the graphicaluser interface is arranged to present the representative visual contentitems of each group that has a minimal number of members above apredefined threshold.

Accordingly, according to an aspect of the present invention, there isprovided a data processing system, wherein the graphical user interfaceis arranged to present the representative visual content items alongsidethe visual content items.

These, additional, and/or other aspects and/or advantages of the presentinvention are: set forth in the detailed description which follows;possibly inferable from the detailed description; and/or learnable bypractice of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will now be described in the following detaileddescription of exemplary embodiments of the invention and with referenceto the attached drawing, in which dimensions of components and featuresshown are chosen for convenience and clarity of presentation and are notnecessarily shown to scale. Generally, only structures, elements orparts that are germane to the discussion are shown in the figure.

FIG. 1 is a schematic illustration of a system setup in accordance withan exemplary embodiment of the invention;

FIG. 2 is a flowchart of acts performed in querying an object, inaccordance with an exemplary embodiment of the invention;

FIG. 3 is a flowchart of acts performed after an object has beenselected, in accordance with an exemplary embodiment of the invention;

FIG. 4 Shows screen shots of the results of exemplary embodiment of theinvention;

FIG. 5 Shows screen shots of the results of the process defined in FIG.6 in accordance with an exemplary embodiment of the invention;

FIG. 6 is a flowchart of acts performed after a query has beensubmitted, in accordance with an exemplary embodiment of the invention;

FIG. 7 Shows screen shots of the menu items and search results inaccordance with an exemplary embodiment of the invention;

FIG. 8 is a flowchart of acts performed in classifying an image into theshapes, in accordance with an exemplary embodiment of the invention;

FIG. 9 is a flowchart of acts performed on each contour collected in, inaccordance with an exemplary embodiment of the invention;

FIG. 10 shows screen shots of screen shot with MVP results in accordancewith an exemplary embodiment of the invention;

FIGS. 11 and 12 are high level flowcharts illustrating a computerimplemented method of running a query item on a plurality of visualcontent items, according to some embodiments of the invention;

FIG. 13 is a scheme describing the system and process in accordance withan exemplary embodiment of the invention;

FIG. 14 is a scheme describing the system in accordance with anexemplary embodiment of the invention;

FIG. 15 is a high level flowchart illustrating a computer implementedmethod of presenting a plurality of visual content items, according tosome embodiments of the invention; and

FIG. 16 is a high level block diagram illustrating a data processingsystem for analyzing and presenting a plurality of visual content items,according to some embodiments of the invention.

DETAILED DESCRIPTION

Provided herein is a detailed description of this invention. It is to beunderstood, however, that this invention may be embodied in variousforms, and that the suggested (or proposed) embodiments are onlypossible implementations (or examples for a feasible embodiments, ormaterializations) of this invention. Therefore, specific detailsdisclosed herein are not to be interpreted as limiting, but rather as abasis and/or principle for the claims, and/or as a representative basisfor teaching one skilled in the art to employ this invention invirtually any appropriately detailed system, structure or manner.

To facilitate understanding the present invention, the followingglossary of terms is provided. It is to be noted that terms used in thespecification but not included in this glossary are considered asdefined according the normal usage of the computer science art, oralternatively according to normal dictionary usage.

The term “GPU” as used herein in this application, is defined as anapparatus adapted to reduces the time it takes to produce images on thecomputer screen by incorporating its own processor and memory, havingmore than 4 CPU's such as GeForce 8800.

The term “API” as used herein in this application, is defined as anapplication program interface that is the software interface to systemservices or software libraries. API may be a third party's source codeinterface that supports requests for service, such as yahoo imageservice API.

The term “URL” as used herein in this application, is defined asUniversal Resource Locator. An Internet World Wide Web Address.

The term “Keypoint” as used herein in this application, is defined asinterest points in an object. For example, in the SIFT framework, theimage is convolved with Gaussian filters at different scales, and thenthe difference of successive Gaussian-blurred images are taken.Keypoints are then taken as maxima/minima of the Difference ofGaussians. Such keypoints can be calculated for the original image orfor a transformation of the original image such as an affine transformof the original images.

The term “Keypoint descriptor” as used herein in this application, isdefined as a descriptor of a key point. For example, in the SIFTframework the feature descriptor is computed as a set of orientationhistograms on neighborhoods. The orientation histograms are relative tothe keypoint orientation and the orientation data comes from theGaussian image closest in scale to the keypoint's scale. Just likebefore, the contribution of each pixel is weighted by the gradientmagnitude, and by a Gaussian with a 1.5 times the scale of the keypoint.Histograms contain 8 bins each, and each descriptor contains an array of4 histograms around the keypoint. This leads to a SIFT feature vectorwith (4×4×8=128 elements).

The term “RGB” as used herein in this application, is defined as anadditive color model in which red, green, and blue light are addedtogether in various ways to reproduce a broad array of colors.

The term “thumbnail” as used herein in this application, is defined as areduced version of an image, which is commonly included in the imagefile itself.

The term “Visual content item” as used herein in this application, isdefined as an object with visual characteristics such as An image filelike BMP, JPG, JPEG, GIF, PNG files; a screenshot; A video file likeAVI, MPG, MPEG, MOV, WMV, FLV files or a one or more frame of a video.

The term “Visual analysis” as used herein in this application, isdefined as the analysis of the characteristics of visual objects such,as visual similarity, coherence, hierarchical organization, concept loador density, feature extraction and noise removal.

The term “Text similarity” as used herein in this application, isdefined as the Measure the pair-wise similarity of strings. Textsimilarity can score the overlaps found between two strings based ontext matching. Identical strings will have a score of 100% while “car”and “dogs” will have close to zero score. “Nike Air max blue” and NikeAir max red” will have a score which is between the two. Other stringsimilarity metrics may also be used.

The term “Labeling” as used herein in this application, is defined asCreating a name for a group of items. For instance in case we areLabeling a product the label will describe several things about theproduct—who made it, when it was made, where it was made, its content,how is it to be use and how to use it safely.

FIG. 1 is a schematic illustration of a system setup 110, which maybenefit from methods in accordance with an exemplary embodiment of theinvention. In setup 110, a server 100 is running an object searchservice. Object can be one of the following: An image file such as BMP,JPG, JPEG, GIF, PNG files; A video file such as AVI, MPG, MPEG, MOV,WMV, FLV files; An audio file such as MP3, WMA, WAV, OGG; A documentsuch as DOC, DOCX, XLS, XML, HTML, PDF files. Optionally, servers suchas server 100, uses one or more GPU's such as GPU 102 to accelerate itscomputations.

Server 100 is connected over a computer network 104 to second server108. Optionally the communication is done through the API 106 of thesecond server 108. The API receives the query parameters, and sends backto server 100 the query results. Query results include a list of objects(for example the object files themselves or a list links to each ofthem). Preferably, such list if significantly longer than the number ofobject that are normally displayed on one page of a typical displaydevice, for example 300 or 500 objects are returned as the laterreordering would usually present a sub set of that list.

Optionally, server 100 has or is connected to GPU 102. Such unitsusually have two advantages: Multiple processors, at the presentcommercially available GPU's have 256 or even 320 stream processorswhile current commercially available Intel processors have 4 processorshence they have an advantage in massively parallel processes. Built inability to accelerate vector operation such as vector additions andsubtractions.

FIG. 2 is a flowchart of acts performed in querying an object, inaccordance with an exemplary embodiment of the invention; and

User input a query 202 (such as a keyword query) to the server 100. Theserver uses API 106 to submit a query request such request containsfields such as: The application ID; The query to search for; The kind ofsearch to submit; The number of results to return; The starting resultposition to return; The finishing position, format (such as bmp, gif,jpeg, png); Whether the service should filter out adult content bydefault; Coloration (such as color, black and white); Site: a domain torestrict searches to; The format for the output; The name of thecallback function to wrap around the data if needed; The API 106responds 204 with fields such as: Result Set: Contains all of the queryresponses. Has attributes, The number of query matches in the database,The number of query matches returned, The position of the first resultin the overall search; Result: Contains each individual response; Title:The title of the image file; Summary: Summary text associated with theimage file; Url: The URL for the image file; Click Url: The URL forlinking to the image file; Referer Url: The URL of the web page hostingthe content; File Size: The size of the file in bytes; File Format: Oneof bmp, gif, jpg, or png; Height: The height of the image in pixels;Width: The width of the image in pixels; Thumbnail: The URL of thethumbnail file and its height and width in pixels; Publisher: Thecreator of the image file.

Restrictions: Provides any restrictions for this media object.Restrictions include noframe and noinline. Noframe means that you shouldnot display it with a framed page on your site. Noinline means that youshould not inline the object in the frame up top. Copyright: Thecopyright owner. Alternatively or additionally, some or all the queryresults can be received from the memory or storage device of server 100.For example they can be retrieved from cached query results saved instage 206 of a previous query. There are at least two possible ways todecide whether to search the query results in the memory or storage: Thequery is identical or similar to a previous query; the links areidentical to links stored in the memory or storage. The Server thenpresents the query results, preferably in the order received on theuser's screen as shown in screen shot 402 of FIG. 4. Optionally, queryresults are than saved 206.

Following that, two processes run in parallel: 208: The objectthumbnails are downloaded from their respective URL's, this processusually starts first 210: Displaying the thumbnails of the object's onthe users display and further information such as the link to theoriginal object and summary data. Alternatively, this process can happenone after another and not in parallel.

FIG. 3 is a flowchart of acts performed after an object has beenselected, in accordance with an exemplary embodiment of the invention.An object is selected 302 by means of mouse click, mouse double click orany other way that enables a user to select it, an example can be shownin the selection of object 410 in FIG. 4. Consequently, the objectsreceived in step 204, such as the objects presented in screen shot 402of FIG. 4, are sorted 304 according to their similarity to the selectedobject.

There are several ways to perform such sorting: Using methods such asScale-invariant feature transform or similar methods such as GLOH(Gradient Location and Orientation Histogram), PCA-SIFT and MSR. Suchmethod usually use keypoint localization step, an later on compare manykeypoint descriptors in one object to a plurality of keypointdescriptors in another object and hence require quick computation inorder to compare an object to a plurality of object within a responsetime an ordinary user would expect. The higher the number or thepercentage of keypoint descriptors in a first object than match (exactlyor approximately) keypoint descriptors in a second object the higher isthe similarity between the two objects. Using methods such as Haarwavelet transform. Comparing the color histograms of the object to othercolor histograms. Categorizing, for example dividing images to imageswith human faces, or human skin vs. other images using face detection ofhuman skin detection software program. The methods can be usedseparately, one after another or in parallel. In case a heavilycomputational method such as (a) is used it is advisable to use a GPUsuch as 102 to attain a reasonable response time.

Once or in parallel to sorting 304 the objects are presented 306 to theuser in descending order of similarity. This way the user can focus onthe objects that are most similar to the selected object. Optionally,objects that have a similarity lower than a certain threshold would notbe presented. The objects compared during process 304 can be either thethumbnails or the object files themselves.

FIG. 4 Shows screen shots of the results of stages 210 and 306. Screenshot 402 shows the results of step 210. It can be seen that the objectare not necessarily sorted according to their similarity to object 410.Once object 410 has been selected by the user, the sort process 304 isperformed. Consequently in step 306 the sorted objects are presented asshown in screen shot 404. Object 410 is shown first and the other objectare presented right and down to him in descending order of similarity.The end result is that once the user has selected object 410, itreceived back only object 410 and its similar objects. This was theprocess of paging back and forth to collect similar objects in an objectsearch engine is saved and higher productivity is achieved.

It is noted that some of the above described embodiments may describethe best mode contemplated by the inventors and therefore may includestructure, acts or details of structures and acts that may not beessential to the invention and which are described as examples.Structure and acts described herein are replaceable by equivalents whichperform the same function, even if the structure or acts are different,as known in the art. Variations of embodiments described will occur topersons of the art. Therefore, the scope of the invention is limitedonly by the elements and limitations as used in the claims, wherein theterms “comprise,” “include,” “have” and their conjugates, shall mean,when used in the claims, “including but not necessarily limited to.”

FIG. 5 Shows screen shots of the results of the process defined in FIG.6 in accordance with an exemplary embodiment of the invention. Screenshot 510 shows a search results page in which the text item “rokefeller”was searched as shown in step 602 of FIG. 6. Since the Faces filter 516has been selected in the category selection area 514, the systemsfilters in items that are most probably faces according to itsparameters as described in steps 612 to 616 of FIG. 6. Hence, the usercan narrow the search to faces.

Screen shot 520 shows a search results page in which the text item“rokefeller” was searched as shown in step 602 of FIG. 6. Since thelandscapes filter 522 has been selected the systems filters in itemsthat are most probably landscapes according to its parameters asdescribed in step 622 of FIG. 6. Hence, the user can narrow the searchto landscapes.

Screen shot 530 shows a search results page in which the text item“phones” was searched as shown in step 602 of FIG. 6. Since the productsfilter 532 has been selected the systems filters in items that are mostprobably products according to its parameters as described in step 632of FIG. 6. Hence, the user can narrow the search to products. In asimilar manner the process described in screen shot can apply todocuments.

Screen shot 540 shows a search results page in which the text item“rockefeller” was searched as shown in step 602 of FIG. 6. Since thecolor filter 542 has been selected the lets the user selects, asdescribed in step 642 of FIG. 6, the dominant color 544 to look for. Thesystem filters in items that are have this dominant color as describedin step 632 of FIG. 6. Hence, the user can narrow the search to objectswith the dominant color 544.

FIG. 6 is a flowchart of acts performed after a query has beensubmitted, in accordance with an exemplary embodiment of the invention.The user queries the system with an object query such as a text query orsuch as the process defined in FIG. 2. The query results are displayed604. The user can than choose 606 to filter the results according toseveral categories such as: Faces 514; Landscapes 522; Products 532; andColor 542; Alternatively the user can choose the filters prior toperforming the query.

The following processes described below assume that a plurality ofobjects has at least one image (such as a thumbnail, or the image fileitself, a video frame) that represents one or more of the object queryresults, and the filters are applied to one or more of those images.

In case a face filter has been chosen 610, a human skin filtering isperformed 612 over the image, and filters in pixels suspected to be ofhuman skin.

A further texture and pixel distribution are further performed 614(before, after or in parallel to 612) to filter images that are mostprobably to include a human face. For example the relative area of thehuman skin divided by the total image are should be above a certainpredefined percentage. Any object that is suspected to include a humanface image is filtered in 616 to be displayed in step 650.

In case a landscapes filter has been chosen 620, a landscape filteringis performed 622 over the images. For example, in an RGB representationof each pixel in each image Blue (=B), Green(=G) and Red (+R)intensities are taken or calculated. In case: B is above threshold b₁;and B/G is above threshold bg₁; and B/R is above threshold br₁; Or incase: B is above threshold b₂; and B/G is above threshold bg₂; and B/Ris above threshold br₂; Than the pixel is considered a landscape pixel.

In the case that in a predefined area of the image the ratio oflandscape pixels divided by the total number of pixels in that areaexceeds a certain threshold, the image is considered a landscape image.

Alternatively, a similar process calculates the ration between blackpixels and the total number of pixels in a predefined area to filter innight time landscape images. In that case a black pixel is defined forexample as a pixel in which: R<tbr₁ and G<tbg₁ and R<tbr₁. Any objectthat is suspected to be landscape image is filtered in 622 to bedisplayed in step 650.

In case a product/document filter has been chosen 630, a landscapefiltering is performed 632 over the images. For example, in an RGBrepresentation of each pixel in each image Blue (=B), Green(=G) and Red(+R) intensities are taken or calculated. In case: B is above thresholdbw₁; and G is above threshold gw₁; and R is above threshold rw₁; Thepixel is considered “white”.

In the case that in a predefined area of the image the ratio of whitepixels divided by the total number of pixels in that area exceeds acertain threshold, the image is considered a product or document image.Any object that is suspected to be product/document image is filtered in632 to be displayed in step 650.

In case a color filter has been chosen 640, the system allows the userto choose 642 between a set of predefined dominant colors as shown initem 544 of FIG. 6.

A dominant color filtering 644 is than performed, For example, in an RGBrepresentation of each pixel in each image Blue (=B), Green(=G) and Red(+R) intensities are taken or calculated. In case the “orange” color hasbeen selected than if: R/G is above threshold O₁; and G/B is abovethreshold O₂; and R is above threshold O₃; The pixel is considered“Orange”.

In the case that in a predefined area of the image the ratio of “Orange”pixels divided by the total number of pixels in that area exceeds acertain threshold, the image is considered an image with a dominantorange color. Any object that is suspected to be an image with adominant orange color is filtered in 644 to be displayed in step 650.

FIG. 7 Shows screen shots of the menu items and search results inaccordance with an exemplary embodiment of the invention. Search screen700 is comprised of: Thumbnails of search results 710; thumbnails ofsearch history 711, such as reduced thumbnails of the images selected inprevious searches; Breadcrumbs-A form of text navigation 712 showing ahierarchical structure of the search history. The current locationwithin the search is indicated by a list of searches before the currentsearch in a hierarchy, leading up to the home page. The following menuitems: Means 722 to select a category such as Faces, Products,Landscapes or “MVP”—most valuable pictures (further explained in FIG.10); Means 724 to further search a particular shape as described inFIGS. 8-9; Means 726 to choose an images format such as portrait,landscape or panoramic in which choosing a format will refine the searchto images with certain range of height/width ratio; means 728 to conductthe search in a certain image database such as Yahoo images, Flickrimages or Picasa; and means 730 to limit the search to a certain licensesuch as creative commons license.

FIG. 8 is a flowchart of acts performed in classifying an image into theshapes, in accordance with an exemplary embodiment of the invention. Instep 802 an image is loaded. Usually the process described in FIG. 8 isperformed at the first time that image is downloaded, but it could bedone in other contexts as well. Subsequent to that color histogram iscomputed 804, than a check is performed 806 for colorfulness—forexample, if a certain color range in the color space exceeds apredefined percentage of the image pixels and the rest of the colorranges are below another predefined threshold than the image isconsidered colorful. If the image is colorful the flow is passed to step808 in which a color segmentation is done—searching the borders of eachcolor cluster, and all the significant contours in the image arecollected. If the image is not colorful the flow is passed to step 810in which an edge detection such as “Sobel” is performed, all connectedcomponents above a certain size collected into contour collection. Then,in step 812 for each of the contours collected by either step 808 or 810the process defined in FIG. 9 is performed.

FIG. 9 is a flowchart of acts performed on each contour collected instep 808 or 810, in accordance with an exemplary embodiment of theinvention. For the contour: A center of the contour is found 902, suchas its center of mass; the contour is transformed 904 into polarcoordinates (r, θ); one or more of the contour coordinates are smoothed906 using methods such as “moving average”, applying various filters, orother smoothing methods. Extremum (a point where a function reaches amaximum or a minimum) points of the contour in polar coordinates arecalculated 908. Further properties of the contour are calculated 910such as its area, the ration between its height and width and itssymmetry. All the calculated parameters are used to classify 912 thecontour into shapes, for example a contour with not extremum points andheight to width ratio of 1 and totally symmetrical is a circle shape.Optionally, the shape information such as number, size, location,rotation texture and color is stored 914. Optionally, the storedinformation is later used for indexing and retrieval 916 of the visualcontent items.

FIG. 10 Shows screen shots of screen shot with MVP results in accordancewith an exemplary embodiment of the invention. Screen shots 1000 iscomprised of: a collection of image thumbnails 1006; a collection 1002of “MVP” images—the dominant images in the current search such as a rowof images; and an indicator 1004 of the number of times each of the MVPobjects appear in the set.

When the MVP control is pressed 722 or after the image set is downloadedor in any other stage, the images in the image set are compared to eachother and similar images (for example, image with close to identicalcolor histogram and/or above a certain two dimensional correlationcoefficient to each other) are collected into clusters. If clusters thancontain exceed a certain number of images are presented in descendingorder of the number of images.

In certain cases in which it is clear that a significant part of theimage set to be presented is comprised of images of a certain categorysuch as: Faces, products, landscapes, images with a dominant red color,cross shapes, or black and white images, this subset will be presentedfirst in its own row such as in 1002.

FIGS. 11 and 12 are high level flowcharts illustrating a computerimplemented method of running a query item on a plurality of visualcontent items, according to some embodiments of the invention. Queryitem may comprise text, images or generally multimedia files, and maycomprise various shapes. The computer implemented method comprises thefollowing stages: analyzing the plurality of visual content itemsaccording to predefined analysis rules relating to visualcharacteristics of the visual content items (stage 120); receiving thequery from a user and identifying at least one query item (stage 125),e.g., using a third party's source code interface that support requestsfor services; searching the visual content items for a plurality ofsuggested visual content items relating to the query items by predefinedcomparison rules (stage 130), suggested visual content items maycomprise thumbnails of selected visual content items; allowing the userto select at least one of the suggested visual content items (stage135); and reordering the visual content items according to theirsimilarity to the selected visual content item and to the analyzing andvisual characteristics of the visual content items (stage 140).

According to some embodiments of the invention, the computer implementedmethod may further comprise applying at least one filter to the visualcontent items (stage 145). The filter may be a landscape filter, a facefilter, a shape filter, product/document filter, or a color filter. Thefilter application (stage 145) may be carried out by using a keypointdescriptors comparison or by using a dedicated graphics renderingdevice.

According to some embodiments of the invention, analyzing the visualcontent items (stage 120) may comprise generating a color histogram,color segmentating for colorful items and detecting edges and editingcontours for non-colorful items (stage 150), wherein items areidentified as colorful or non-colorful according to the color histogramand at least one predefined threshold. Analyzing the visual contentitems (stage 120) may additionally or alternatively comprise applying atwo dimensional correlation analysis (stage 155). Analyzing the visualcontent items (stage 120) may comprise analyzing the content itemsaccording to shapes occurring in the query items.

According to some embodiments of the invention, the computer implementedmethod may further comprise removing content items according to apredefined removal criteria (stage 160) such as an absence of the atleast one shape from the query items.

According to some embodiments of the invention, the computer implementedmethod may further comprise transforming the shapes to polar coordinates(stage 165), finding extremum points in contours of the shapes, andusing the properties of the extremum points to classify the contour intoshape categories (stage 170).

According to some embodiments of the invention, the computer implementedmethod may further comprise counting and presenting the number of thereordered visual content items (stage 175).

According to some embodiments of the invention, the computer implementedmethod may further comprise applying at least one operator on the visualcontent items to receive modified visual content items (stage 180).Reordering the visual content items (stage 140) may be carried out infurther relation to the modified visual content items. Reordering thevisual content items (stage 140) may be carried out by classifying thevisual content items to predefined categories relating to the queryterms and the analysis of the visual content items.

FIG. 13 is a scheme describing the system and process in accordance withan exemplary embodiment of the invention. System 1300 performs theprocess described hereinafter: A person 1302 captures using a capturingmeans 1301 a visual object of a tangible object 1304. The visual objectis sent over a network 1306 such as the internet to a processing system1308. Processing system comprises multiple processing units configuredto allow larger scale processing. A preferably multi-core processorssystem 1308 runs the grouping algorithm described in FIG. 3. Partial orfull results are sent over network such as 1306 to a user terminal 1310.User terminal 1310 displays the results end user 1312.

FIG. 14 is a scheme describing the system in accordance with anexemplary embodiment of the invention. System 1400 is a display showing:A first subpart 1402 showing several content items, in this case productphotos with their prices and titles. A second subpart showingsimultaneously 10 clusters. Each of this clusters was calculated usingcluster analysis of the content items: A major part of the objects werecompared to each other and similar objects were collected to the samecluster. The top 10 clusters are presented by their number of members indescending order, next to each of them their respective number ofelements is presented. A third subpart 1406 showing the number 1407 ofobjects containing a significant area of each of the predefined 19colors. A fourth subpart 1408 showing the number 1409 of objectscontaining a shapes area of each of the predefined set of shapes.

FIG. 14 is a scheme describing the system in accordance with anexemplary embodiment of the invention. System 1400 is a display showing:A first subpart 1402 showing several content items, in this case productphotos with their prices and titles. A second subpart 1410 showingsimultaneously 10 clusters. Each of this clusters was calculated usingcluster analysis of the content items: A major part of the objects werecompared to each other and similar objects were collected to the samecluster. The top 10 clusters 1404 are presented by their number ofmembers in descending order, next to each of them their respectivenumber of elements is presented. Selecting each of them such as clickingon 1401 will result in presenting only the content items that belong toits group. Each of the groups can display the number of its members 1405and have a label 1403 as described in stage 225. A third subpart 1406showing the number 1407 of objects containing a significant area of eachof the predefined 19 colors. A fourth subpart 1408 showing the number1409 of objects containing a shapes area of each of the predefined setof shapes; selecting a representative visual content item for each group(stage 225), such selection uses the visual items parameters such asusing each object's visual match to other items in group, its resolutionsymmetry, background uniformity, optionally the group is labeled usingthe text fields of its items; and presenting the representative visualcontent item of each group that has a minimal number of members above apredefined threshold (stage 230). The computer implemented method mayfurther comprise presenting the plurality of visual content itemsalongside the representative visual content items (stage 235). Thecomputer implemented method may further comprise storing groupinginformation for later caching for a certain period of time (fixed orcalculated by the relative change in the items set) of said results(stage 237). The computer implemented method may further allowing formanual change of the groups such as deletion, insertion, subdivision ormerger of groups (stage 239).

According to some embodiments of the invention, the range may be a fixednumber. According to some embodiments of the invention, grouping (stage220) may be carried out by using predefined color groups, or predefinedshape groups, or any other categorization. Grouping (stage 220) may becarried out using at least one keypoint of the visual content items. Atleast one of the groups may comprise human faces, product images,landscape images, or any other item category. The visual content itemsmay be product offerings, for example such that are presented on anonline market place.

According to some embodiments of the invention, the representativevisual content items of each group may be presented in descending orderof their range for the number of its members. The representative visualcontent items may be reduced images, such as thumbnails, and may bepresented on a predefined sub part of a users' display, or on a separatewindow.

FIG. 16 is a high level block diagram illustrating a data processingsystem 280 for analyzing and presenting a plurality of visual contentitems 262, according to some embodiments of the invention. Dataprocessing system 280 comprises a mediator server 250 comprising agraphical user interface 260. Mediator server 250 is connected via acommunication link 271 with users 270 and via a communication link 241with a plurality of sources holding the visual content items 240.Mediator server 250 is arranged to group visual content items 262according to predefined similarity rules relating to visualcharacteristics of visual content items 262 such that each group has arange for the number of its members; and to select a representativevisual content item 266 for each group. Graphical user interface 260 isarranged to present representative visual content items 266 of eachgroup that has a minimal number of members above a predefined threshold.Graphical user interface 260 may be arranged to present representativevisual content items 266 in a predefined subarea 264 of the display,alongside visual content items 262 presented in a different subarea 261.A further subarea 267 may be allocated for user selectable categories269.

According to some embodiments of the invention, the range may be a fixednumber. According to some embodiments of the invention, grouping may becarried out by using predefined color groups, or predefined shapegroups, or any other categorization, relating to user selectablecategories 269 or unrelated thereto. Grouping may be carried out inadvance, responsive to user selections and queries, or on the flydynamically on the inventory of visual content items 262. Grouping maybe carried out by mediator server 250 using at least one keypoint of thevisual content items. At least one of the groups may comprise humanfaces, product images, landscape images, or any other item category.Visual content items 262 may be product offerings, for example such thatare presented on an online market place. According to some embodimentsof the invention, representative visual content items 266 of each groupmay be presented in descending order of their range for the number ofits members. Representative visual content items 266 may be reducedimages, such as thumbnails.

According to some embodiments of the invention, a data processing systemfor running a query item on a plurality of visual content items ispresented. The data processing system comprises a mediator server(hosting API 106) connected via a communication link (e.g., internet104) with a user and with a plurality of sources holding the visualcontent items (through web server 100 with GPU 102), and arranged toanalyze the plurality of visual content items according to predefinedanalysis rules relating to visual characteristics of the visual contentitems; to receive the query from the user and to identify at least onequery item therein; to search the visual content items for a pluralityof suggested visual content items relating to the query items bypredefined comparison rules; to allow the user to select at least one ofthe suggested visual content items; and to reorder the visual contentitems according to their similarity to the selected visual content itemand to the analyzing and visual characteristics of the visual contentitems.

According to some embodiments of the invention, the systems and methodsgroups similar images into the same group, and then presets the majorgroups in descending order, larger groups first. According to someembodiments of the invention, the systems and methods download thesystem feeds on a daily basis, decide on a list on main views, analyzethe feeds to calculate the product clusters and preparing output filesfor the vendors. According to some embodiments of the invention, thesystems and methods have the advantages of: No overload—Instead ofreading the whole book, a “table of content” window is presented on theleft side that shows the major product groups. Apart from functionallyit reduces the current visual load of the current loads on text in thecurrent layout; The navigation is not linear! Clicking on a productgroups will take the shopper to that group, no need to page down. Visualinterface—no need for prior familiarity with intricate productcategories—everything has a picture; thus, one click will select yourdesired product group and what you see is what you get. Pareto—Theproduct groups are presented in descending order of importance,according to their relative “market share” which is the number ofproducts in each group. Additional advantages are: Userexperience—improved user experience; Faster Navigation to the desiredproduct that increases conversion rates and decreases the load on thesystem; and Positioning—position the system as using best of breadshopping technology.

Regarding color, according to some embodiments of the invention, thesystems and methods uses an economic 19 natural color palette ratherthan using an artificial color palette, which may better cater toshoppers than an RGB palette. Not using natural colors creates anotherproblem, though one color is chosen, other color appear. Separatingproduct from its background, in many product photos, the product appearswith a background. Context—the systems and methods are contextsensitive, only the colors that appear in the offering are shown, and anumber shows the number of deals which contain, the relevant color. Thisway shopper can see the color distribution of the relevant offering andfocus on the existing colors. The systems and methods may work online oroffline. Offline, they may analyze content items from providers inrelation to their form, color and content and convert the results to astandardized file. Actual purchases may them be related or included inthe files. Online, users may search the offers using queries that may belikewise analyzed.

In the above description, an embodiment is an example or implementationof the inventions. The various appearances of “one embodiment,” “anembodiment” or “some embodiments” do not necessarily all refer to thesame embodiments.

Although various features of the invention may be described in thecontext of a single embodiment, the features may also be providedseparately or in any suitable combination. Conversely, although theinvention may be described herein in the context of separate embodimentsfor clarity, the invention may also be implemented in a singleembodiment.

Reference in the specification to “some embodiments”, “an embodiment”,“one embodiment” or “other embodiments” means that a particular feature,structure, or characteristic described in connection with theembodiments is included in at least some embodiments, but notnecessarily all embodiments, of the inventions.

It is to be understood that the phraseology and terminology employedherein is not to be construed as limiting and are for descriptivepurpose only.

The principles and uses of the teachings of the present invention may bebetter understood with reference to the accompanying description,figures and examples.

It is to be understood that the details set forth herein do not construea limitation to an application of the invention.

Furthermore, it is to be understood that the invention can be carriedout or practiced in various ways and that the invention can beimplemented in embodiments other than the ones outlined in thedescription above.

It is to be understood that the terms “including”, “comprising”,“consisting” and grammatical variants thereof do not preclude theaddition of one or more components, features, steps, or integers orgroups thereof and that the terms are to be construed as specifyingcomponents, features, steps or integers.

If the specification or claims refer to “an additional” element, thatdoes not preclude there being more than one of the additional element.

It is to be understood that where the claims or specification refer to“a” or “an” element, such reference is not be construed that there isonly one of that element.

It is to be understood that where the specification states that acomponent, feature, structure, or characteristic “may”, “might”, “can”or “could” be included, that particular component, feature, structure,or characteristic is not required to be included.

Where applicable, although state diagrams, flow diagrams or both may beused to describe embodiments, the invention is not limited to thosediagrams or to the corresponding descriptions. For example, flow neednot move through each illustrated box or state, or in exactly the sameorder as illustrated and described.

Methods of the present invention may be implemented by performing orcompleting manually, automatically, or a combination thereof, selectedsteps or tasks.

The term “method” may refer to manners, means, techniques and proceduresfor accomplishing a given task including, but not limited to, thosemanners, means, techniques and procedures either known to, or readilydeveloped from known manners, means, techniques and procedures bypractitioners of the art to which the invention belongs.

The descriptions, examples, methods and materials presented in theclaims and the specification are not to be construed as limiting butrather as illustrative only.

Meanings of technical and scientific terms used herein are to becommonly understood as by one of ordinary skill in the art to which theinvention belongs, unless otherwise defined.

The present invention may be implemented in the testing or practice withmethods and materials equivalent or similar to those described herein.

Any publications, including patents, patent applications and articles,referenced or mentioned in this specification are herein incorporated intheir entirety into the specification, to the same extent as if eachindividual publication was specifically and individually indicated to beincorporated herein. In addition, citation or identification of anyreference in the description of some embodiments of the invention shallnot be construed as an admission that such reference is available asprior art to the present invention.

While the invention has been described with respect to a limited numberof embodiments, these should not be construed as limitations on thescope of the invention, but rather as exemplifications of some of thepreferred embodiments. Other possible variations, modifications, andapplications are also within the scope of the invention. Accordingly,the scope of the invention should not be limited by what has thus farbeen described, but by the appended claims and their legal equivalents.

What is claimed is:
 1. A computer implemented method of presenting aplurality of visual content items, comprising: receiving a text querysubmitted by a user; grouping a plurality of visual content items whichare provided as query results to said text query in a plurality ofgroups according to a plurality of predefined similarity rules relatingto a plurality of visual characteristics of the plurality of visualcontent items; selecting a plurality of representative visual contentitems each for one group of the plurality of groups, each one of saidplurality of representative visual content-items is selected accordingto at least one image parameter thereof, each one of said plurality ofrepresentative visual content items is selected from the plurality ofvisual content items of a respective said group; presenting a graphicaluser interface having: a first subpart presenting said plurality ofrepresentative visual content items each of one of the plurality ofgroups; a second subpart presenting a plurality of categorization itemseach indicative of a different color; a third subpart presenting membersof one of the plurality of groups; and filtering at least some of saidplurality of groups or at least some of said plurality of visual contentitems in said plurality of groups in response to at least one userselection of at least one of said categorization items in said secondsubpart; updating a presentation of said graphical user interface on adisplay of a user terminal in response to an outcome of said filtering;labeling each group of the plurality of groups with a group labelselected from a plurality of text fields of members of the respectivegroup; wherein the group label describes at least one non visualcharacteristic of the respective representative visual content item;wherein said presentation comprises presenting said group label; whereinthe plurality of visual content items are images of a plurality ofproducts and the least one non visual characteristic in indicative of amember of a group consisting of; who made a product, when a product wasmade, where a product was made, a content of a product, a product usageinstructions, and a safely of a product.
 2. The computer implementedmethod of claim 1, further comprising presenting the plurality of visualcontent items alongside the representative visual content item.
 3. Thecomputer implemented method of claim 1, wherein at least one of saidplurality of predefined similarity rules defines a fixed number ofvisual content items.
 4. The computer implemented methods of claim 1,wherein the grouping is carried out by using predefined color groupsindicated by said to at least one user selection.
 5. The computerimplemented methods of claim 1, wherein the grouping is carried outusing predefined shape groups indicated by said to at least one userselection.
 6. The computer implemented method of claim 1, wherein atleast one of the groups comprises human faces.
 7. The computerimplemented method of claim 1, wherein at least one of the groupscomprises product images.
 8. The computer implemented method of claim 1,wherein at least one of the groups comprises landscape images.
 9. Thecomputer implemented method of claim 1, wherein the representativevisual content item of each group are presented in descending order oftheir range for the number of its members.
 10. The computer implementedmethod of claim 1, wherein the representative visual content item ofeach group are reduced images.
 11. The computer implemented method ofclaim 1, wherein the representative visual content item of each groupare thumbnails.
 12. The computer implemented method of claim 1, whereinthe representative visual content item are presented on a predefined subpart of a users' display.
 13. The computer implemented method of claim1, wherein the representative visual content item are presented on aseparate window.
 14. The computer implemented method of claim 1, whereinthe grouping is carried out using at least one keypoint of the visualcontent items.
 15. The computer implemented method of claim 1, whereinthe visual content items are product offerings.
 16. The computerimplemented method of claim 1, wherein the visual content items areproduct offerings presented on an online market place.
 17. The method ofclaim 1, wherein members other than the representative visual contentitem are presented in a descending order of similarity relative to therepresentative visual content item.
 18. The method of claim 1, whereinthe grouping is performed according to text similarity between textstrings which are associated with different visual content items fromthe plurality of visual content items.
 19. The method of claim 1,further comprising: selecting a first group of the plurality of groupsby a user which clicks on an area in the presentation; presenting aplurality of members of the first group in response to the selecting.20. The method of claim 1, wherein the plurality of visual content itemsare image files of a plurality of webpages.
 21. The computer implementedmethod of claim 1, wherein said at least one image parameter is selectedfrom a group consisting of resolution, symmetry, and backgrounduniformity.
 22. The computer implemented method of claim 1, wherein eachof said plurality of categorization items is indicative of saiddifferent color and a shape category.
 23. A data processing system foranalyzing and presenting a plurality of visual content items,comprising: a mediator computer server adapted to be connected via onecommunication link to a user client and another communication link to aplurality of sources holding the plurality of visual content items;wherein said mediator computer server is adapted to: receive a textquery submitted by a user; group the plurality of visual content itemswhich are provided as query results to said text query in a plurality ofgroups according to a plurality of predefined similarity rules relatingto a plurality of visual characteristics of the plurality of visualcontent items such that each group of the plurality of groups has arange for the number of its members, select a plurality ofrepresentative visual content items each for one group of the pluralityof groups, each one of said plurality of representative visual contentitems is selected according to at least one image parameter thereof,each one of said plurality of representative visual content items isselected from the plurality of visual content items of a respective saidgroup; present a graphical user interface having: a first subpartpresenting said plurality of representative visual content items; asecond subpart presenting a plurality of categorization items eachindicative of a different color; a third subpart presenting members ofone of the plurality of groups; and filter at least some of saidplurality of groups or at least some of said plurality of visual contentitems in said plurality of groups in response to at least one userselection of at least one of said categorization items in said secondsubpart; update a presentation of said graphical user interface on adisplay of a user terminal in response to an outcome of said filtering;label each group of the plurality of groups with a group label selectedfrom a plurality of text fields of members of the respective group;wherein the group label describes at least one non visual characteristicof the respective representative visual content item; wherein saidpresentation comprises presenting said group label; wherein theplurality of visual content items are images of a plurality of productsand the least one non visual characteristic in indicative of a member ofa group consisting of; who made a product, when a product was made,where a product was made, a content of a product, a product usageinstructions, and a safely of a product.
 24. The data processing systemof claim 23, wherein the graphical user interface is arranged to presentthe representative visual content items alongside the visual contentitems of the respective group.
 25. The data processing system of claim23, wherein for each group, members other than the representative visualcontent item are presented in a descending order of similarity relativeto the representative visual content item.